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ABSTRACT 

The  world  in  which  governments  must  make  policy  is  changing  rapidly  in  unpredictable 
ways.  Changes  in  information  and  communication  technologies  are  eroding  national  borders 
and  creating  global  markets.  The  relationship  between  the  public  and  private  sectors  is 
changing.  Furthermore,  because  of  the  globalization  of  issues  and  the  interrelationships 
among  systems,  the  consequences  of  making  wrong  policy  decisions  are  becoming  more 
serious  -  even  catastrophic.  What  do  these  developments  imply  for  policymaking  and  policy 
analysis?  This  talk  suggests  that  a  new  policymaking  paradigm  and  new  tools  for  performing 
policy  analysis  are  needed  that  will  help  governments  to  deal  with  uncertainty  in  policy 
formulation  and  implementation.  The  proposed  paradigm,  called  adaptive  policymaking, 
provides  a  flexible  and  rapid  response  to  changing  circumstances  and  builds  in  a  plan  for 
learning  over  time  as  part  of  the  decisionmaking  process.  An  adaptive  policy  would  take  those 
actions  now  that  cannot  be  deferred,  prepare  to  take  actions  that  may  later  become  necessary, 
monitor  changes  in  the  world,  and  take  actions  when  the  monitoring  process  shows  they  are 
needed.  New  policy  analysis  tools  based  on  massive  computing  can  be  used  by  policy  analysts 
to  support  the  formulation  of  adaptive  policies. 


I.  INTRODUCTION 

I  grew  up  in  the  United  States  in  the  middle  of  the  20th  century.  The  1950s  were  very 
conservative  times  in  the  United  States.  World  War  II  was  over,  the  country  was  experiencing 
unprecedented  economic  growth  and  prosperity.  Industry  began  to  mass-produce  an  immense 
variety  of  new  goods  and  technological  gadgets  to  provide  the  post-war  generation  with 
comfort  and  entertainment.  Television  came  into  its  own.  The  TV  dinner  epitomized  the 
lifestyle.  And  gas-guzzling  automobiles  with  shiny  chrome  grilles  epitomized  the  culture. 

The  Cold  War  was  on.  The  military  knew  where  the  next  war  would  be  fought  -  in  the  Fulda 
Gap,  along  the  Elbe,  between  East  Germany  and  West  Germany.  So,  all  of  the  West’s  military 
planning  was  focused  on  preventing  this  specific  next  war,  or  on  winning  it.  It  would  be  hard 
to  overestimate  the  amount  of  time  and  resources  that  were  devoted  to  analyzing  this  single 
scenario.  Similarly,  corporate  strategic  planning  was  based  on  stability  in  the  environment. 
RCA  spent  almost  10  years  developing  the  first  commercially  available  television  set.  Du 
Pont  worked  for  12  years  to  get  nylon  into  production. 

These  were,  indeed,  simpler,  stable  times.  People  could  make  long-term  plans  for  their  lives. 
Corporations  offered  well-defined  career  paths;  employees  could  plan  on  lifetime  jobs  with  a 
single  company.  Defense  planning  focused  on  a  single  scenario,  policymakers  could  put  into 
place  long-term  policies  with  expectations  that  they  would  be  long-lasting,  and  policy  analysts 
could  use  predictive  models  to  perform  ‘’what-if  analyses,  and  talk  realistically  about  making 
‘optimal’  decisions. 
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II.  PROBLEM  STATEMENT 

This  well-ordered,  stable  environment  began  to  change  dramatically  in  the  late  1960s.  Some 
ascribe  this  change  to  the  Beatles;  some  to  the  Vietnam  war.  I  was  in  college  during  this 
transitional  period.  Bob  Dylan  told  us  “the  times  they  are  a-changing”  -  they  were,  and  they 
have  continued  to  do  so,  at  what  seems  to  be  an  accelerating  rate. 

Rapid  advances  in  information  technology  have  been  the  catalysts  for  many  of  these  changes. 
The  power  of  computers  has  been  doubling  every  18  months,  and  the  amount  of  traffic  being 
carried  over  the  Internet  has  been  doubling  every  100  days.  The  cost  of  a  telephone  call  from 
the  United  States  to  London  declined  by  90  percent  in  the  last  ten  years,  during  which  time 
the  consumer  price  index  rose  by  over  40  percent.  Such  developments  have  led  to  a  sea 
change  in  the  structure  of  human  society,  the  nature  of  business  and  industry,  and  the  role  of 
government. 

The  economy  is  now  a  global  economy.  For  a  significant  and  growing  portion  of  economic 
activity,  distance  no  longer  determines  the  boundaries  of  sales  markets,  the  set  of  competitors, 
or  even  labor  markets.  In  the  past,  white-collar  jobs  in  traditional  service  industries  were 
located  near  a  company’s  head  office.  Now,  such  jobs  can  be  competed  for  by  any  country  in 
the  world  that  has  a  decent  telecommunications  system  and  an  appropriately  skilled  labor 
force.  National  boundaries  make  little  difference  to  corporations  or  to  financial  markets,  and 
markets  can  be  quite  fickle.  Starbucks  is  driving  local  cappuccino  shops  out  of  business  in 
Italy.  Trillions  of  dollars  flow  around  the  world  every  day.  Nations  are  powerless  to  fight  such 
strong  market  forces. 

Given  an  interlinked  global  economy  and  powerful  market  forces,  how  does  a  national 
government  protect  its  own  national  interests?  Countries  have  long  wielded  power  and  control 
over  what  went  on  within  their  borders.  (A  physical  reminder  of  this  power  is  that  whatever 
entered  the  country  was  monitored  by  customs  officers  at  borders  and  airports.)  Cyberspace  is 
eroding  those  borders.  Intellectual  properties  flow  freely  across  the  Internet,  knowing  no 
borders.  It  is  next  to  impossible  for  a  nation  to  monitor  them,  much  less  block  or  levy  taxes  on 
them.  Not  only  is  internal  monitoring  and  control  becoming  more  difficult,  but  what  happens 
externally  now  has  an  increasing  influence  on  what  happens  internally.  For  example,  in  the 
past,  companies  could  focus  attention  on  local  or  national  markets  and  governments  could 
focus  on  their  own  economies.  Success  within  these  borders  was  normally  sufficient  for 
survival,  and,  indeed,  for  prosperity.  This  is  no  longer  true. 

Furthermore,  because  of  the  globalization  of  issues  and  the  interrelationships  among  systems, 
the  consequences  of  making  wrong  decisions  have  become  more  serious  -  even  catastrophic. 
The  economic  success  of  countries  in  Euroland  is  dependent  on  the  decisions  made  by  a  few 
persons  at  the  European  Central  Bank.  Every  day  we  hear  dire  warnings  about  the  possible 
negative  effects  of  introducing  genetically  modified  organisms  into  our  food  chain.  And  I 
don’t  need  to  tell  you  what  the  consequences  of  global  warming  might  be  for  the  Netherlands. 
The  real  problem  in  most  of  these  potentially  catastrophic  situations  is  that  scientific 
knowledge  has  not  yet  advanced  sufficiently  to  offer  definitive  answers.  But  policy  decisions 
have  to  be  made  -  and  are  being  made  -  about  these  issues. 

What  do  these  developments  imply  for  policymaking  and  policy  analysis?  I  believe  that  they 
imply  the  need  for  a  new  policymaking  paradigm  and  new  tools  for  performing  policy 
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analysis  that  will  enable  governments  to  deal  with  uncertainty  in  policy  formulation  and 
implementation.  I  will  discuss  these  implications  in  the  remainder  of  this  talk. 

III.  TRADITIONAL  POLICY  ANALYSIS 

First,  for  those  of  you  who  are  unfamiliar  with  public  policy  analysis,  I  will  present  a  very 
short  explanation  of  traditional  policy  analysis  and  how  it  has  developed  and  changed  over  the 
past  half-century  to  respond  to  changes  in  government  and  society.  Policy  analysis  is  a 
rational,  systematic  approach  to  making  policy  choices  in  the  public  sector.  It  supports 
policymakers  in  identifying  policies  that  are  cost-effective  and  that  would  help  them  to 
achieve  their  policy  goals  and  objectives.  I  see  policy  analysis  as  a  bridge  between  the 
scientific  community  and  the  policy  community.  The  policy  community  often  suffers  because 
so  little  of  what  goes  on  in  the  scientific  community  is  made  understandable  and  accessible  to 
it.  Policy  analysts  act  as  translators  and  bridges  between  these  two  communities. 

The  traditional  policy  analysis  process  generates  information  on  the  consequences  that  would 
follow  the  adoption  of  various  policies.  (This  is  sometimes  referred  to  as  “what  if’  analysis.) 
It  uses  a  variety  of  tools  to  develop  this  information  and  to  present  it  to  the  parties  involved  in 
the  policymaking  process  in  a  manner  that  helps  them  come  to  a  decision.  (For  a  description 
of  the  traditional  policy  analysis  process,  and  an  example  of  its  application  to  a  problem 
regarding  the  growth  of  air  transport  in  the  Netherlands,  see  [Walker,  2000].) 

Policy  analysis  has  its  roots  in  a  variety  of  scientific,  engineering,  and  applied  mathematics 
disciplines.  It  is  not  a  specific  methodology,  but  it  makes  use  of  a  variety  of  methodologies  in 
order  to  solve  specific  public  policy  problems.  It  evolved  from  operations  research  (in  the  late 
1940s  and  early  1950s)  through  systems  analysis  (in  the  late  1950s  and  early  1960s)  to  policy 
analysis  as  a  result  of  problem-oriented  work  for  governments  carried  out  at  the  RAND 
Corporation  and  other  applied  research  organizations  in  the  1960s  and  1970s.  In  the 
beginning,  operations  research  techniques  were  applied  to  problems  in  which  there  were  few 
parameters  and  a  clearly  defined  single  objective  function  to  be  optimized  (e.g.,  aircraft 
design  and  the  placement  of  radar  installations).  Gradually,  the  problems  being  analyzed 
became  broader  and  the  contexts  more  complex.  Health,  housing,  transportation,  and  criminal 
justice  policies  were  being  analyzed.  Single  objectives  (e.g.,  cost  minimization  or  single 
variable  performance  maximization)  were  replaced  by  the  need  to  consider  tradeoffs  among 
multiple  (and  conflicting)  objectives  (e.g.,  the  impacts  on  health,  the  economy,  and  the 
environment,  and  the  distributional  impacts  on  different  social  or  economic  groups).  Non- 
quantifiable  and  subjective  considerations  had  to  be  considered  in  the  analysis.  Optimization 
was  replaced  by  what  Herb  Simon  (1969,  pp.  64-65)  called  satisficing.  He  defined  satisficing 
to  mean  finding  an  acceptable  or  satisfactory  solution  to  a  problem  instead  of  an  optimal 
solution.  He  said  that  satisficing  was  necessary  because  "in  the  real  world  we  usually  do  not 
have  a  choice  between  satisfactory  and  optimal  solutions,  for  we  only  rarely  have  a  method  of 
finding  the  optimum." 

The  broadening  of  problems  and  contexts  led  to  a  broadening  of  the  disciplinary  expertise 
required  to  perform  policy  analysis.  The  pioneers  of  policy  analysis  focused  on  military 
systems  and  were  primarily  physical  scientists,  mathematicians,  and  engineers.  However,  in 
order  to  deal  with  social  problems,  such  as  health  and  housing,  social  and  behavioral  scientists 
were  needed.  Each  of  these  added  disciplines  brought  new  perspectives  and  new  tools  that  are 
being  used  to  improve  and  enrich  the  way  public  policies  are  identified,  discussed,  evaluated, 
chosen,  and  implemented. 
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Policy  analysts  have  also  expanded  their  focus  well  beyond  the  analysis  of  alternative  policies 
to  consider  the  entire  policymaking  process.  It  has  been  increasingly  realized  that  the 
appropriate  involvement  of  citizens  and  stakeholder  groups  in  the  problem-solving  process 
can  profoundly  affect  policy  outcomes  and  popular  support  for  those  outcomes. 

Furthermore,  because  of  the  need  to  deal  with  problems  in  our  rapidly  changing,  complex, 
unpredictable  world,  uncertainty  has  become  an  increasingly  important  element  in  the 
analysis.  There  are  many  types  of  uncertainties  that  policy  analysts  (and  policymakers)  have 
to  deal  with  -  e.g.,  uncertainty  about  model  form,  uncertainty  about  the  values  of  a  model’s 
parameters,  and  uncertainty  about  underlying  probability  distributions.  Mathematicians, 
statisticians,  modelers,  and  others  have  devoted  a  great  deal  of  time  and  effort  to  most  of  these 
types,  and  have  developed  effective  tools  for  dealing  with  them.1  But,  today  I  want  to  focus 
on  a  type  of  uncertainty  that  I  call  structural  uncertainty,  for  which  we  have  developed  few 
tools.  Structural  uncertainty  relates  to  the  future  structural  elements  of  the  world  that  are 
relevant  for  making  policy,  but  are  unknown  and  unknowable  at  the  time  of  the  analysis  -  for 
example,  which  countries  will  be  most  powerful  in  2030,  how  will  the  population  be 
distributed  between  cities  and  outlying  areas,  how  will  the  climate  change?  The  answers  for 
2030  will  be  known  with  certainty  in  2030  -  but  will  remain  uncertain  until  then.  As  we  get 
closer  to  2030,  the  uncertainties  will  decrease.  But,  decisions  have  to  be  made  in  the  face  of 
structural  uncertainty.  For  example,  because  of  long  lead  times,  we  need  to  be  thinking  now 
about  infrastructure  changes  to  water  management  systems  and  transport  systems  that  will  be 
able  to  handle  the  problems  to  be  faced  in  2030.  How  can  this  be  done? 

IV.  TRADITIONAL  WAYS  OF  DEALING  WITH  UNCERTAINTY 

Up  until  now,  decisionmakers  have  dealt  with  structural  uncertainty  in  one  of  two  ways.  The 
first  (and  most  common)  is  to  ignore  it  -  to  overlook  it  or  act  as  if  it  is  not  there.  An  implicit 
assumption  is  made  that  the  future  world  will  be  structurally  more  or  less  the  same  as  the 
current  world  -  perhaps  more  populated,  richer,  dirtier  -  but,  essentially  the  same.  Of  course, 
this  does  not  solve  the  uncertainty  problem.  It  merely  sweeps  it  under  the  rug,  and  can  have 
serious  consequences.  For  example,  as  discussed  in  a  paper  by  de  Neufville  [2000],  the 
telephone  company  of  France  was  a  pioneer  in  the  use  of  on-line  interactive 
telecommunications.  It  committed  itself,  on  the  basis  of  the  most  careful  analyses,  to  the 
development  of  the  Minitel  system.  But,  it  failed  to  build  in  the  capability  to  change  as  the 
world  changed  —  to  expand  to  more  advanced  platforms  using  improved  technologies  for  the 
system.  This  resulted  in  a  network  that  is  now  virtually  obsolete  in  the  Internet  environment, 
and  that  cannot  practically  be  adapted  to  the  new  technical  realities.  It  is  a  dinosaur  less  than 
20  years  after  its  initiation. 

The  second  approach  to  dealing  with  structural  uncertainty  is  more  enlightened.  It 
corresponds  to  the  current  policymaking  paradigm  and  forms  the  basis  for  traditional  ‘what-if’ 
policy  analysis.  It  worked  fairly  well  in  the  past  when  change  was  more  gradual  and 
predictable,  there  was  less  global  competition,  and  the  consequences  of  being  wrong  were 
smaller.  The  central  assumption  of  this  paradigm  is  that  the  future  can  be  predicted  well 
enough  to  identify  policies  that  will  produce  favorable  outcomes  in  one  or  more  specific 
plausible  future  worlds.  The  future  worlds  are  called  scenarios.  Policy  analysts  use  best- 
estimate  models  (based  on  the  most  up-to-date  scientific  knowledge)  to  examine  the 
consequences  that  would  follow  from  the  implementation  of  each  of  several  possible  policies 


1  See,  for  example,  [Cooke,  1991]  and  [Morgan  and  Henrion,  1990]. 
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in  each  scenario.  The  ‘best’  policy  is  the  one  that  produces  the  most  favorable  outcomes 
across  the  scenarios.  (Such  a  policy  is  called  a  robust  policy.)  The  problem  with  this  approach 
is  that  the  resulting  policy  is  best  for  specific  scenarios  that  are  fairly  certain  not  to  occur, 
since  any  given  scenario  has  a  probability  zero  of  actually  occurring.  More  important,  the 
resulting  policy  has  implications  for  the  future  that  actually  occurs  that  were  probably  not 
examined  in  the  course  of  the  analysis  and  that  are  generally  not  revisited  as  the  future 
unfolds. 

This  approach  has  had  its  successes  in  the  past  and  can  work  quite  well  -  in  fact,  its  popularity 
can  be  traced  to  its  success  in  helping  the  Shell  Oil  Company  handle  the  oil  crisis  in  the  early 
1970s.  However,  if  a  policy  is  based  on  a  variety  of  assumptions  about  the  future  and  some  of 
those  assumptions  turn  out  to  be  wrong,  the  negative  consequences  can  be  as  bad  as  if 
structural  uncertainty  had  been  totally  ignored.  Consider  the  recent  case  of  planning  for  the 
future  of  Schiphol  Airport.  In  1995,  after  a  two-year  multi-phased  deliberative  process  known 
as  “physical  planning  key  decision  Schiphol”  (PKB  Schiphol),  some  major  decisions  were 
made  by  the  Dutch  Parliament  that  were  intended  to  guide  the  growth  of  civil  aviation  in  the 
Netherlands  to  the  year  2015. 

One  of  the  outcomes  of  the  PKB-Schiphol  process  was  the  decision  to  constrain  the  number 
of  passengers  at  Schiphol  to  no  more  than  44  million  passengers  per  year.  This  constraint  was 
supposed  to  be  more  than  enough  to  accommodate  the  most  optimistic  estimates  of  passenger 
growth  until  at  least  the  year  2015.  These  limits  will  certainly  be  reached  well  before  then. 
And  the  noise  limits,  also  expected  to  be  reached  no  sooner  than  2015,  were  reached  in  1999. 

How  did  such  a  long,  costly,  and  deliberate  planning  process  do  such  a  poor  job  in  forecasting 
the  growth  in  air  traffic  at  Schiphol?  The  passenger  and  noise  projections  were  based  on 
passenger  forecasts  that  were  produced  by  a  model  developed  by  the  Central  Planning  Bureau 
[Centraal  Planbureau,  1992].  This  model  assumes  that  the  number  of  passengers  passing 
through  Schiphol  is  directly  related  to  the  value  of  the  Netherlands’  Gross  National  Product 
(GNP).  This  assumption  was  based  upon  the  fact  that,  up  until  the  time  the  model  was  built, 
there  had  been  a  very  close  relationship  between  the  GNP  and  the  number  of  passengers 
passing  through  Schiphol. 

Of  course,  no  one  knows  with  certainty  what  the  GNP  will  be  in  2015.  So,  the  CPB 
developed  three  scenarios,  each  with  a  different  value  of  GNP,  which  were  then  used  to 
produce  three  forecasts  of  the  number  of  passengers  at  Schiphol  in  2015.  The  44  million 
figure  corresponds  to  the  forecast  based  on  the  highest  GNP  growth  rate  of  the  three 
scenarios.  The  actual  growth  of  GNP  through  1999  was  closest  to  the  assumptions  in  the  low- 
growth  scenario.  Nonetheless  (as  shown  in  Figure  1)  the  growth  in  the  number  of  passengers 
during  this  period  was  significantly  more  than  what  was  forecast  using  the  assumptions  from 
the  high-growth  scenario  —  called  Balanced  Growth. 

What  happened  was  that  a  number  of  trend  breaks  -  unanticipated  changes  in  the  world  of 
civil  aviation  -  had  occurred  after  the  forecasts  had  been  made.  The  forecasts  had  assumed 
that  the  future  would  be  a  continuation  of  the  past.  But,  in  fact,  three  factors  that  have  little  to 
do  with  GNP  growth  rates  were  responsible  for  the  rapid  growth  of  air  traffic  at  Schiphol: 

1 .  The  growth  of  hub-and-spoke  networks,  with  Schiphol  becoming  a  hub  airport  for 
KLM,  where  it  cross-connects  transfer  passengers  whose  destination  is  not 
Amsterdam,  but  some  other  KLM  city.  Most  of  the  growth  in  passenger  traffic  through 
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Schiphol  has  come  from  an  increase  in  the  number  of  transfer  passengers  carried  by 
KLM.  (The  transfer  traffic  at  Schiphol  grew  from  27%  in  1990  to  43%  in  1998.) 

2.  A  code-sharing  alliance  between  KLM  and  Northwest  Airlines,  which  feeds 
Northwest’s  European  traffic  through  KLM,  and  therefore  through  Schiphol. 

3.  The  European  Union’s  decision  to  liberalize  the  air  transport  industry  -  to  reduce 
national  monopolies  and  increase  competition  among  airlines.  As  a  result  of  this 
decision,  European  airlines  are  facing  competitive  pressures  that  they  did  not  have  to 
face  in  the  past,  fares  have  fallen,  and  the  demand  for  air  travel  has  increased. 

So,  great  care  must  be  taken  in  developing  and  using  scenarios  to  deal  with  structural 
uncertainty.  It  is  absolutely  crucial  to  consider  trend  break  scenarios. 


Schiphol  Passenger  Traffic 
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Figurel  -  Actual  and  Projected  Growth  of  Passenger  Traffic  at  Schiphol  Airport  (1990-2000) 


V.  A  NEW  POLICYMAKING  PARADIGM 

I  now  turn  to  a  third  approach  to  dealing  with  structural  uncertainty  in  policy  formulation  and 
implementation.  This  approach  will  require  a  new  policymaking  paradigm  that  will  enable 
policymakers  to  react  to  rapidly  changing  situations.  It  is  based  upon  the  following  line  of 
reasoning: 

(1)  In  this  unpredictable  rapidly  changing  world,  it  is  almost  impossible  to  identify  robust 
policies  -  fixed  static  policies  that  will  perform  well  against  all  plausible  futures. 

(2)  Over  time,  we  gain  information  that  resolves  current  structural  uncertainties. 

(3)  Thus,  the  best  policies  will  be  adaptive  (take  those  actions  now  that  cannot  be 
deferred;  prepare  to  take  actions  that  may  later  become  necessary;  monitor  changes  in 
the  world  and  take  actions  when  they  are  needed) 

Most  current  policies  do  not  explicitly  provide  for  flexibility  of  response  to  changing 
circumstances,  and  do  not  incorporate  a  plan  for  learning  over  time  as  part  of  the 
decisionmaking  process.  When  events  occur  that  invalidate  some  of  the  underlying 
assumptions  of  a  policy,  the  policy  may  remain  unchanged  for  a  long  time  or  policymakers 
may  scramble  to  quickly  develop  new  policies.  This  approach  is  popularly  called  ‘muddling 
through’.  Adaptive  policymaking  and  adaptive  policies  would  explicitly  recognize  that 
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policies  have  to  be  adjusted  as  the  world  changes  and  as  new  information  becomes  available. 
The  suggested  ‘adaptive’  approach  would  create  policies  in  advance  that  would  respond  to 
changes  over  time  and  that  would  make  explicit  provision  for  learning.  As  a  result,  the 
inevitable  policy  changes  would  become  part  of  a  larger,  recognized  process  and  would  not 
have  to  be  made  repeatedly  on  an  ad-hoc  basis. 

The  concept  of  adaptation  is  easy  to  explain.  It  is  analogous  to  the  approach  used  in  guiding  a 
ship  through  a  long  ocean  voyage.  The  goal  -  the  end  point  -  is  set  at  the  beginning  of  the 
journey.  It  remains  constant.  But,  along  the  way,  unpredictable  storms  and  other  traffic  may 
interfere  with  the  original  trajectory.  So,  the  policy  -  the  specific  route  -  is  changed  along  the 
way.  It  is  understood  before  the  ship  leaves  port  that  some  changes  are  likely  to  take  place  - 
and  contingency  plans  may  have  already  been  formulated  for  some  of  the  unpredictable 
events.  The  important  thing  is  that  the  ultimate  goal  remains  unchanged,  and  the  policy 
actions  implemented  over  time  remain  directed  toward  that  goal.  An  adaptive  policy  would 
include  a  systematic  method  for  monitoring  the  environment,  gathering  information, 
implementing  pieces  of  the  policy  over  time,  and  adjusting  and  re-adjusting  to  new 
circumstances.  The  policies  themselves  would  be  designed  to  be  incremental,  adaptive,  and 
conditional. 

Guiding  the  ship  of  state  in  this  adaptive  way  may  be  revolutionary  in  many  policy  areas. 

But,  it  is  already  an  accepted  approach  in  some.  For  example,  the  U.S.  Federal  Open  Market 
Committee  and  the  European  Central  Bank  set  inflation  targets  and  then  change  interest  rates 
as  conditions  in  the  economy  change.  It  is  also  rapidly  becoming  a  guiding  principle  -  indeed, 
a  requirement  for  survival  -in  the  private  sector. 

The  analysis  and  choice  of  an  adaptive  policy  would  require  a  new  process  for  policymaking 
that  explicitly  takes  into  account  the  uncertainties  and  dynamics  of  the  problem  being 
addressed.  Two  colleagues  of  mine  and  I  have  designed  an  approach  to  adaptive 
policymaking  [Walker,  Cave,  and  Rahman,  2001].  We  divide  the  process  into  two  phases:  a 
‘thinking  phase’  and  an  ‘implementation  phase.’  In  the  thinking  phase  the  policy  problem  is 
formulated,  the  policy  analysis  is  conducted,  and  the  adaptive  policy  is  specified,  including 
the  rules  for  its  implementation.  The  implementation  phase  consists  of  the  actual  sequence  of 
events  and  actions  that  represent  the  execution  of  the  previously  agreed  upon  adaptive  policy. 

I  will  briefly  present  the  process  to  you  today  as  food  for  thought. 

Thinking  Phase 

The  first  step  in  the  adaptive  policymaking  process  is  a  ’stage-setting’  exercise  designed  to 
make  policy  goals  explicit,  develop  a  clear  set  of  options,  and  construct  a  definition  of  policy 
success  that  makes  operational  sense.  Successive  steps  in  the  process 

•  assemble  the  structure  of  the  policy, 

•  identify  key  uncertainties, 

•  separate  actions  to  be  taken  now  from  those  that  can  or  should  be  deferred  until 
more  information  becomes  available, 

•  develop  signposts  for  monitoring  changes  in  the  world  and  triggers  for  contingency 
plans,  and 

•  establish  limits  to  the  validity  of  the  analysis  that,  once  violated,  should  lead  to  a 
reassessment  of  the  policy. 
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Figure  2  presents  a  diagrammatic  view  of  the  adaptive  policymaking  process.  I  will  now  walk 
you  through  the  figure,  describing  the  major  steps  in  the  process,  illustrating  them  using  a 
simplified,  semi-hypothetical  example.2  Suppose  that  Schiphol  Airport  is  running  out  of 
capacity  and  will  not  be  able  to  accommodate  much  more  growth.  The  central  policy 
questions  are:  Should  demand  be  accommodated?  If  so,  how?  If  not,  how  best  to  limit  it? 


Figure  2  -  The  Adaptive  Policymaking  Process 


Step  1:  Setting  the  Stage 

The  activities  in  the  rounded  box  in  the  upper-left  comer  of  Fig.  I  constitute  the  stage-setting 
step  in  the  policymaking  process.  In  this  step,  the  objectives,  constraints,  and  available  policy 
options  are  specified.  This  results  in  a  definition  of  success  and  policy  goals.  For  our 
example,  suppose  the  goal  was  to  maintain  a  major  role  for  the  Netherlands  in  civil  aviation. 
The  constraints  could  be  those  imposed  by  noise,  emissions,  economic,  and  other 
considerations.  And  the  policy  options  could  include  various  ways  of  increasing  airport 
capacity  and  a  host  of  measures  for  shaping  demand,  increasing  efficiency,  and  mitigating  the 
adverse  side-effects  of  a  growth  in  air  transport. 

Step  2:  Assembling  the  Basic  Policy 

The  next  step  is  assembly  of  a  basic  policy  -  a  well-defined  initial  policy  together  with  the 
conditions  that  must  be  met  in  order  for  it  to  succeed.  Continuing  the  example,  a  basic  policy 
could  be  developed  around  the  expansion  of  Schiphol  Airport.  It  might  include  the  building  of 
a  new  runway,  plus  additional  measures  for  reducing  noise  around  the  airport.  Necessary 
conditions  for  success  of  this  policy  might  be  that  the  demand  for  air  transport  continues  to 
increase,  Schiphol  maintains  its  market  share  compared  to  competing  airports,  and  public  (or 
political)  acceptance  of  noise  does  not  decrease. 


2  For  a  more  complete  presentation  of  this  example,  see  [RAND  Europe,  1997]. 
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Step  3:  Specifying  the  Rest  of  the  Policy 

In  the  third  step  of  the  adaptive  policymaking  process,  the  necessary  conditions  for  success 
and  the  details  of  the  basic  policy  are  combined  in  two  sorts  of  forward-looking  analyses, 
which  result  in  a  specification  of  the  remaining  pieces  of  the  policy.  One  analysis  is  the 
identification  of  vulnerabilities  -  both  uncertain  and  certain  adverse  consequences  of  the  basic 
policy.  These  vulnerabilities  can  reduce  acceptance  of  the  policy  to  the  point  where  its  success 
is  jeopardized.  Two  uncertain  vulnerabilities  of  the  basic  policy  in  the  Schiphol  example 
might  be  that:  KLM  is  acquired  by  another  airline  or  goes  bankrupt,  leaving  Schiphol  without 
a  hub  airline,  and  accidents  increase.  In  anticipation  of  an  uncertain  vulnerability,  such  as  an 
increased  risk  of  accidents,  various  hedging  actions  can  be  developed  to  diversify  or  reduce 
exposure  or  cushion  the  consequences.  In  our  example  these  might  include  subsidizing 
accident  insurance. 

A  certain  vulnerability  of  the  basic  policy  would  be  an  increase  in  noise  around  Schiphol 
from  a  growth  in  air  transport.  Associated  with  the  certain  vulnerabilities  are  mitigating 
actions  to  be  put  in  place  immediately.  These  might  include  subsidizing  sound  insulation  or 
creating  ‘noise  markets.’ 

The  second  analysis  is  the  translation  of  the  necessary  conditions  for  success  into  signposts 
that  should  be  monitored  in  order  to  be  certain  that  the  underlying  analysis  remains  valid,  that 
implementation  is  proceeding  on  schedule  and  according  to  expectations,  and  that  necessary 
policy  corrections  or  additional  actions  are  taken  in  a  timely  and  effective  manner.  The 
identification  of  signposts  does  not  call  for  immediate  implementation  of  any  direct  policy 
actions.  But  once  signposts  are  identified,  efforts  must  be  initiated  to  collect  and  monitor  the 
necessary  information.  Critical  values  of  the  signpost  variables,  called  triggers,  are  specified 
in  this  step.  These  will  lead  to  the  implementation  of  appropriate  contingency  plans,  which 
should  also  be  specified  in  this  step.  Signposts  in  the  Schiphol  example  might  include 
monitoring  the  growth  of  demand  for  air  transport  and  monitoring  the  health  of  KLM  (e.g.,  its 
market  share  and  profitability).  Slower  than  expected  growth  in  demand  might  trigger  a  delay 
in  the  expansion  of  Schiphol.  If  KLM  shows  signs  of  failing,  actions  might  be  taken  to  prop  it 
up  or  to  attract  another  hub  airline  to  Schiphol.  In  line  with  the  idea  of  signposts  and  triggers, 
Mr.  John-Hugh  Rees  of  the  European  Commission  Directorate  General  for  Transport  and 
Energy  recently  called  for  a  new  approach  to  collecting  and  using  statistical  information  for 
decisionmaking.  In  particular  he  said  “there  will  ...  be  an  increasing  need  for  rapidly  available 
indicators  that  flag  .  .  .  changes  that  need  the  early  attention  of  policymakers”  [Rees,  2000]. 

Implementation  Phase 

Once  the  policy  --  including  signposts,  triggers,  mitigating  actions,  etc.—  is  agreed  upon,  we 
are  in  the  implementation  phase.  In  this  phase,  events  unfold  and  signpost  information  is 
collected.  The  adaptive  policymaking  process  (as  distinct  from  the  adaptive  policy  itself)  is 
suspended  until  a  trigger  event  is  reached. 

Once  the  policy  is  agreed  upon,  we  are  in  the  implementation  phase.  In  this  phase,  events 
unfold  and  signpost  information  is  collected.  The  adaptive  policymaking  process  (as  distinct 
from  the  adaptive  policy  itself)  is  suspended  until  a  trigger  event  is  reached. 

The  response  to  a  trigger  event  depends  on  the  nature  of  the  alarm.  Many  contingencies  will 
have  been  foreseen  in  the  original  plan.  As  long  as  the  basic  policy,  objectives,  and 
constraints  remain  in  place,  these  responses  can  be  characterized  as  defensive  actions  or 
corrective  actions.  In  our  example,  a  major  air  disaster  may  overwhelm  the  hedging  and 
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mitigating  provisions,  and  lead  to  widespread  dissatisfaction.  The  basic  policy  could  then  be 
defended  by  publicizing  impending  safety  improvements,  or  corrected  by  scaling  back 
expansion  plans.  If  the  trigger  is  slower  than  expected  demand  growth,  some  combination  of 
reduction  in  the  planned  expansion  or  direct  demand  subsidies  might  be  employed. 

Under  some  circumstances,  neither  defensive  nor  corrective  actions  will  suffice.  For  instance, 
there  may  be  major  changes  in  stakeholders’  objectives,  extremely  large  shocks  to  signpost 
information  (e.g.,  a  collapse  of  demand,  runaway  growth  in  demand,  rapid  growth  in  regional 
air  traffic),  or  significant  unforeseen  actions  by  other  players  (e.g.,  a  large  cooperative 
expansion  of  Charles  de  Gaulle  and  Berlin  airports).  In  such  cases,  the  policy  should  be  re¬ 
examined  in  its  entirety.  The  conditions  triggering  a  reassessment  should  be  stated  explicitly 
as  part  of  the  original  policy  (just  as  the  conditions  for  corrective  and  defensive  actions  are 
part  of  the  policy).3 

This  means  restarting  the  policymaking  process.  But  the  process  would  not  have  to  be  started 
‘from  scratch.’  When  the  policy  is  reassessed,  much  more  will  be  known  about  the  world  and 
the  identities,  motivations,  and  capabilities  of  other  key  players.  Also,  many  aspects  of  the 
policy  will  already  be  in  place  and  more  will  be  known  about  the  effects  of  the  initial  policies. 
Finally,  participants  in  the  process  are  likely  to  have  a  significant  collective  commitment  to 
the  process  as  a  whole. 

To  sum  up,  adaptive  policy  analysis  looks  both  from  the  present  to  the  future  and  from  the 
future  to  the  present,  in  order  to  develop  ways  of  comparing  where  we  are  going  to  where  we 
would  like  to  go.  It  views  uncertainty  as  something  to  be  qualitatively  understood  in  order  to 
manage  the  timing  of  critical  decisions  and  to  develop  robust  policies.  And  it  is  continuously 
making  use  of  new  information  to  resolve  the  original  uncertainties  over  time. 

IV.  WORKING  WITH  THE  NEW  POLICYMAKING  PARADIGM 

Developing  and  implementing  adaptive  policies  will  not  be  easy.  There  are  significant  legal, 
political,  and  analytic  barriers  to  be  overcome.  The  legal  and  political  barriers  are  likely  to  be 
the  hardest  to  overcome.  Existing  laws  have  been  built  up  over  the  years  to  support  the  current 
approach  to  policymaking  and  policy  execution.  The  organization  and  operation  of  existing 
institutions,  both  governmental  and  non-governmental,  are  based  on  the  current  paradigm. 
Even  if  there  were  overwhelming  support  for  the  new  paradigm,  it  could  not  be 
operationalized  overnight,  but  would  require  the  efforts  of  large  numbers  of  experts  from  a 
variety  of  disciplines.  Nevertheless,  it  should  be  easier  to  implement  in  a  country  like  the 
Netherlands,  whose  government  is  based  on  consensus  and  stability,  than  in  the  United  States, 
where  a  new  government  is  often  inclined  to  reverse  the  policies  of  its  predecessor. 

It  will  also  require  new  analytical  tools  to  help  policy  analysts  identify  good  adaptive  policies. 
I  believe  that  recent  increases  in  computing  power  can  provide  the  basis  for  such  tools.  The 
growth  in  computer  power  over  the  past  decade  has  resulted  in  computing  capabilities  that 
have  never  existed  before.  But,  policy  analysts  have  been  slow  to  capitalize  on  these  new 
capabilities.  Our  approaches  are  still  based  on  methods  that  were  developed  when  we  had 
only  limited  computing  power  (e.g.,  using  few  scenarios  or  performing  sensitivity  analysis 
around  a  single  base  case).  We  policy  modelers  have  always  known  that  our  models  were 


3  Davis,  Gompert,  and  Kugler  [1996]  discuss  a  similar  approach  to  adaptive  policymaking.  They  call  all  but  the  reassessment 
‘operational  adaptiveness’  and  the  built  in  trigger  for  reassessment  ‘strategic  adaptiveness’. 
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simplifications  of  reality  -  and  not  accurately  predictive.  They  are  often  valid  only  under 
fairly  restrictive  assumptions.  For  most  practical  problems,  many  of  these  assumptions  are 
not  true.  Similarly,  as  I’ve  already  shown,  scenario-based  approaches  that  focus  on  optimizing 
results  for  a  given  scenario  may  lead  to  serious  negative  consequences  if  the  assumptions 
prove  untrue.  But,  these  tools  have  continued  to  be  used  (with  improvements  around  their 
edges)  because  there  were  no  viable  alternatives. 

The  availability  of  cheap,  powerful  computing  capability  is  now  providing  an  alternative.  The 
combination  of  computational  speed,  graphical  display,  and  data  handling  make  modem 
computers  qualitatively  different  from  the  computers  available  in  the  1950s  when  current 
policy  analysis  tools  were  developed.  It  is  now  becoming  possible  to  explore  the  full 
implications  for  policy  choices  of  structural  uncertainties.  Rather  than  limiting  our  analysis  to 
a  single  ‘best  estimate’  future  or  two  or  three  scenarios  we  can  explore  the  implications  of 
millions  of  plausible  future  possibilities.  The  implications  for  policy  analysis  are  profound.  In 
particular,  massive  computing  can  supply  the  analytic  support  for  the  development  of  adaptive 
policies. 

Research  to  develop  modeling  tools  for  harnessing  this  computer  power  and  to  develop  new 
approaches  to  policy  analysis  based  on  these  tools  is  currently  being  carried  out  by  RAND 
researchers  in  California  under  the  labels  “exploratory  modeling”  [Bankes,  1993]  and 
“exploratory  analysis”  [Davis  and  Hillestad,  2000].  The  idea  is  to  use  the  computer  in  the  way 
that  laboratories  have  long  been  used  by  physical  scientists.  Instead  of  physical  experiments, 
we  would  make  computational  experiments.  In  fact,  we  would  use  computational  power  and 
graphical  data  visualization  tools  to  ran  thousands  or  millions  of  computational  experiments 
and  make  their  results  comprehensible. 

In  the  process  of  constructing  an  exploratory  computer  model,  a  large  number  of  assumptions 
are  made.  Running  such  a  model  reveals  how  the  system  would  behave  if  those  assumptions 
were  correct.  We  would  systematically  explore  how  the  outcomes  of  interest  would  change  as 
a  function  of  circumstances  and  assumptions,  looking  for  answers  to  questions  such  as: 

-  Under  what  circumstances  would  the  policymakers’  goals  be  reached  or  success 
achieved? 

-  Under  what  circumstances  would  a  policy  fail  or  lead  to  negative  consequences? 

-  What  are  the  threshold  values  that  would  suggest  changes  in  policy  decisions? 

-  Under  which  of  several  adaptive  strategies  would  long-term  outcomes  be  acceptable 
for  a  wide  range  of  futures? 

For  instance,  this  approach  might  be  used  to  determine  the  conditions  under  which 
government  investments  in  a  specific  transport  infrastructure  project  could  be  justified,  those 
for  which  they  could  not  be  justified,  and  those  for  which  we  are  unable  to  tell.  In  a  defense 
context,  Davis  [1994]  described  the  use  of  “scenario  space  analysis”  to  explore  the  entire 
plausible  scenario  space.  His  objective  was  to  increase  the  fraction  of  the  scenario  space  in 
which  certain  stated  objectives  would  be  achieved,  as  opposed  to  identifying  good  policies  for 
a  few  arbitrary  scenarios. 

In  short,  exploratory  modeling  and  exploratory  analysis  could  be  used  to  support  the  type  of 
adaptive  policymaking  I  have  described. 

The  concepts  of  exploratory  modeling,  exploratory  policy  analysis,  and  adaptive  policy 
analysis  may  sound  a  bit  vague  and  theoretical.  In  order  to  make  them  more  concrete,  let  me 
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show  how  Lempert,  Schlesinger,  and  Bankes  [1996]  have  applied  them  in  the  context  of 
devising  a  policy  to  respond  to  the  (highly  uncertain)  situation  of  global  warming. 

Figure  3  is  a  notional  diagram  that  depicts  the  major  conclusions  from  their  study.  It  shows 
how  the  best  strategies  vary  throughout  a  scenario  space  in  which  the  key  factors  X,  Y,  and  Z 
are  unknown  and  uncontrollable.  (X  is  the  probability  that  extreme  damages  will  result  from 
an  increase  in  temperature.  Y  is  the  probability  that  ways  will  be  found  to  reduce  the  cost  of 
abating  greenhouse  gas  emissions.  Z  is  the  sensitivity  of  global  temperature  to  an  increase  in 
greenhouse  gases.)  The  question  is,  under  what  assumptions  about  X,  Y,  and  Z  would  various 
strategies  prove  superior.  Strategy  A  is  one  in  which  a  lot  is  done  right  away  to  reduce  carbon 
emissions.  Strategy  B  is  one  of  wait  and  see  before  doing  anything.  The  adaptive  strategy 
implements  a  series  of  actions  as  new  information  becomes  available.  The  researchers 
explored  the  outcomes  that  could  be  expected  in  the  complete  (X,  Y,  Z)  space  (which 
involved  about  40,000  runs  of  a  series  of  four  inter-linked  models).  They  concluded  that  a 
strategy  of  do-a-lot-right-away  would  be  wise  only  if  one  believed  that  the  consequences  of 
temperature  change  were  likely  to  be  quite  severe  (the  top  right  comer  of  the  figure).  A 
strategy  of  wait-and-see  would  be  wise  only  if  the  consequences  of  temperature  change  were 
likely  to  be  quite  modest  (bottom  left  corner  of  the  figure).  But,  in  the  very  large  region  in 
between,  an  adaptive  strategy  that  does  more  or  less  over  time  depending  on  the  observed 
changes  in  global  temperature  is  superior. 


Source:  Davis  and  Hillestad,  2000. 
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Figure  3  -  The  relative  effectiveness  of  strategies  in  a  3-dimensionsaI  scenario  space 

Using  these  results,  it  would  be  possible  to  change  the  focus  of  the  climate  change  policy 
debate.  It  is  now  a  battle  between  those  who  favor  investing  heavily  now  to  reduce  carbon 
emissions  and  those  who  want  to  wait  and  see  what  might  happen  to  climate  in  the  future.  The 
focus  could  be  changed  to  devising  an  adaptive  strategy  that  would  increase  or  decrease  the 
rate  of  investments  in  low-carbon  methods  over  time  as  a  function  of  signposts  and  triggers 
related  to  temperature  change.  This  would  change  the  frame  surrounding  the  policy  problem 
from  one  of  resolving  disagreements  among  scientists  that  are  unresolvable  given  the  current 
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state  of  knowledge  to  one  of  preparing  for  unpredictable  contingencies.  Within  the  new  frame 
we  could  prepare  an  adaptive  policy  under  which  actions  would  be  taken  now  that  cannot  be 
deferred  and  a  system  of  signposts  and  triggers  would  be  set  up  to  monitor  changes  in  the 
world  and  to  take  actions  when  needed.  This  would  allow  policymakers  to  make  reasonable 
choices  now  about  climate  change  policy  without  requiring  accurate  or  widely  accepted 
predictions  of  the  future. 


V.  CONCLUDING  REMARKS 

Where  does  this  all  leave  us? 

We  must  be  flexible,  adaptive,  open  to  change,  but  prepared  for  whatever  the  future  might 
bring.  There  is  no  scenario  I  could  have  thought  up  that  would  have  led  to  the  trajectory  of  my 
life  from  its  beginning  to  where  I  am  now.  From  the  Bronx  in  1942,  to  Queens  in  1951,  to 
Cornell  University  in  1959,  to  Stanford  University  in  1964,  back  to  Cornell  in  1965,  to  the 
New  York  City-RAND  Institute  in  1970,  to  RAND  Santa  Monica  in  1977,  to  RAND  Europe 
in  1 994,  and  now  to  a  Professor  in  Delft.  The  world  has  been  continuously  changing  around 
me.  But,  to  an  amazing  extent,  I  have  kept  approximately  my  same  professional  goals  -  at 
least  since  my  graduate  school  days  in  the  early  1 960s,  when  I  decided  to  devote  my  life  to 
applying  mathematics  and  modeling  to  solving  real  world  public  policy  problems.  If  anything, 
I  have  become  more  and  more  of  a  missionary  for  policy  analysis.  I  am  delighted  that  the  seed 
of  policy  analysis  that  was  planted  in  Delft  when  I  was  here  in  1 989  has  blossomed  into  the 
faculty  of  Technology,  Policy  and  Management,  and  that  I  have  now  been  honored  by  an 
invitation  to  become  a  member  of  that  faculty.  I  never  dreamed  of  that  outcome.  But,  I  am 
grateful  for  it. 

And  so,  to  my  bottom  line  message  for  policy  analysis  -  and,  in  fact  for  all  of  us  -  as  we 
begin  the  21st  century.  It  comes  from  a  most  unlikely  source  -  a  19th  century  English 
naturalist  named  Charles  Darwin.  His  answer  to  the  question  “who  survives?”  is: 

Not  the  strongest, 

not  the  most  intelligent, 

but  those  who  are  most  responsive  to  change. 
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